The present invention relates to a network monitoring system and method for monitoring the activity on a network carrying message packets where each of the message packets contains source and destination addresses.
Traffic monitoring is a vital element of network and system management. Very little happens in a networked enterprise without producing some network traffic. Monitoring this traffic provides important information about the operation of enterprise applications. This information is essential for activities such as cost allocation, capacity planning, quality of service analysis, fault detection and isolation and security management.
Traffic monitoring used to be a relatively straightforward task. In the past, a large number of machines was connected to a single shared network. A single instrument connected to the network could monitor all the traffic. Requirements for increased bandwidth, changes in traffic patterns, and the quickly falling price of packet switching and routing devices has caused a rapid movement away from shared networks to networks which are highly segmented. The challenge is to monitor traffic on these segmented networks. The use of point-to-point links makes it difficult to attach monitoring instrument to each of the network segments. Furthermore, the large number of instruments required to monitor all the segmented parts of the network ensures that such an approach would not be cost effective. In addition, because the switches and routers themselves have complex internal architectures, the flow of packets within, and through, them is becoming an important factor in network performance.
Because of the need to monitor this modern network configuration, a number of approaches to monitoring network traffic have been developed by different companies:
Hewlett-Packard's (TM) Extended RMON uses packet sampling as a way of monitoring shared local area network (“LAN”) segments. An interface connected to the network is operated promiscuously in order to observe all the packets on the segment. A statistical sampling entity samples packets based on packet count and only the sampled packets are analyzed. While this approach is suited to monitoring network segments, it has limitations as a technique for monitoring traffic within a switching device. First, operating interfaces promiscuously imposes an additional load on the switching device (e.g. switch or router) since it would otherwise filter out a large fraction of the traffic on the segment and only concern itself with the packets that need to be switched or routed. Second, knowing the traffic on the network segment provides no information about the amount of traffic that is entering the switching device or about the resources within the switching device that are being used to handle the traffic. Finally, this approach monitors switched traffic twice, once at the input port, and once at the output port.
Network Instruments' (TM) Observer scans each interface on a switch, monitoring all traffic on an interface for a short period before moving on to the next interface of the same switch. This is a time-based sampling system, and time-based sampling has been shown to be less accurate than packet count based sampling. In addition, the Observer system does not provide for remote analysis of the time-based samples. The samples are analyzed by an instrument attached to the switch, or embedded within the switch. The cost of this instrument adds significant cost to the switch.
Cisco System (TM) routers, as part of their NetFlow monitoring system, send information about completed traffic flows, to a central collector. The router is required to maintain a list of active packet flows, updated by every new packet. This technique requires considerable internal resources from the router or switch. Adding NetFlow to a device adds significantly to its cost, especially in the low end and mid-range markets. This technique also suffers from delay problems. Completed flows may represent traffic that is many minutes old. While this is acceptable for accounting purposes, it does not provide timely information for congestion management.